Skip to content

std.Build.Watch: key fanotify file descriptors by mount id #24113

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jun 23, 2025

Conversation

kevinboulain
Copy link
Contributor

I hope it's okay to take over the inactive pull request #20672. The patch is very similar but I've taken into account the review comments: since indices correspond between the 'public' DirTable and 'private' (to the Linux implementation) HandleTable I've stashed the mount point information in there. However, the Windows implementation is quite similar and shares markAllFilesDirty so I had to add a small edge case, not sure what you'll think of it.
I've been using Zig for the past two weeks so apologies if I'm not following conventions. I've tested my change by following the wiki and since I'm on NixOS I'm hitting this problem immediately (same error message as #20670).

Fixes #20670.
Closes #20672.

Since marks are limited to a single filesystem.
Context: ziglang#20670
Original pull request: ziglang#20672

Co-authored-by: Maciej 'vesim' Kuliński <vesim809@pm.me>
@andrewrk andrewrk merged commit 640a130 into ziglang:master Jun 23, 2025
10 checks passed
@andrewrk
Copy link
Member

Nice work, thanks!

@Profpatsch
Copy link

Profpatsch commented Jul 13, 2025

I’m still getting the error because it re-mounts /nix/store as read-only bind-mount here and fanotify does not like that:

> ~/kot/zig/zls/result/bin/zig build check -
fincremental --watch
Build Summary: 2/2 steps succeeded
check success
└─ compile exe sqlitefs-check Debug native success 362ms
error: unable to watch /nix/store/0vkz54zg4lmfr6rzqkmnznp2mfckb4r3-zig-0.15.0-dev.936+fc2c1883b/lib/std: NotSameFileSystem
error: the following build command failed with exit code 1:
.zig-cache/o/1c616aff675c3ca94f65a3d541a2f645/build /nix/store/0vkz54zg4lmfr6rzqkmnznp2mfckb4r3-zig-0.15.0-dev.936+fc2c1883b/bin/zig /nix/store/0vkz54zg4lmfr6rzqkmnznp2mfckb4r3-zig-0.15.0-dev.936+fc2c1883b/lib /home/philip/depot/users/Profpatsch/sqlitefs .zig-cache /home/philip/.cache/zig --seed 0xf78133c2 -Z10a067d96f509605 check -fincremental --watch
> mount
…
/dev/mapper/cryptroot on /nix/store type btrfs (ro,relatime,ssd,space_cache,subvolid=666,subvol=/root)
/dev/mapper/cryptroot on / type btrfs (rw,relatime,ssd,space_cache,subvolid=666,subvol=/root)
…

@kevinboulain
Copy link
Contributor Author

kevinboulain commented Jul 13, 2025

(I'm out for the next few weeks so I won't be able to take a closer look for now.)

I'm not sure I understand your setup with the information you've pasted. Do I understand correctly that your source files are under /nix/store (Zig's stdlib) and your project, its dependencies and cache all are under /home/philip (i.e.: Zig's local cache, your project sources ~/kot/zig/zls, and what appears to be dependencies /home/philip/depot/users/Profpatsch/sqlitefs)?
Then I assume that neither your /home nor anything under it is a separate mountpoint and it's all part of / (as in /dev/mapper/cryptroot on /)?
Maybe you could confirm that the mount ID is the same for /nix/store and / then? That would mean it's not enough to serve as a key.

@Profpatsch
Copy link

How do I see the mount ID for a mountpoint?

@Profpatsch
Copy link

Ah, after reading the manual: /proc/self/mountinfo

31 1 0:28 /root / rw,relatime shared:1 - btrfs /dev/mapper/cryptroot rw,ssd,space_cache,subvolid=666,subvol=/root
30 31 0:28 /root/nix/store /nix/store ro,relatime shared:14 - btrfs /dev/mapper/cryptroot rw,ssd,space_cache,subvolid=666,subvol=/root

So the bind-mount has a different MountID

@Profpatsch
Copy link

The error description of EXDEV in fanotify_mark is quite

EXDEV The filesystem object indicated by pathname resides within a filesystem subvolume (e.g., btrfs(5)) which uses a different fsid than its root superblock. This error can be returned only with an fanotify group that identifies filesystem objects by file handles.

But I think it gets thrown because I’m using btrfs, and as always btrfs is a special little snowflake when it comes to syscall behaviour on Linux. Maybe a solution here could be to create a different watch group?

@Profpatsch
Copy link

I think I figured it out; I wasn’t aware that fanotify was a relatively recent kernel API, and btrfs support was only added in 6.8 (fsnotify/fsnotify#114 (comment)). I’m currently on 6.6 still.

SO I think the way to do this is to catch that EXDEV error and provide an error message that includes a hint, or alternatively fall back to the inotify interface.

@kevinboulain
Copy link
Contributor Author

If it helps to pinpoint the root cause on your side, I've tested this patch with Zig's stdlib in the Nix store on its own Btrfs subvolume and my project's source directory on another Btrfs subvolume, and, IIRC, under Linux 6.15. That's why I asked about your setup because yours doesn't seem to actually involve multiple filesystems so it's a bit confusing you're hitting this error (it might not even be necessary to key by mount id since it seems to be the same underlying filesystem in your case).
The issue you've linked does sound like the likely culprit and if it is, I think your suggestion

SO I think the way to do this is to catch that EXDEV error and provide an error message that includes a hint

is fair since there's already a precedent:

error.UnsupportedFlags => fatal("fanotify_init failed due to old kernel; requires 5.17+", .{}),

@Profpatsch
Copy link

Okay, so, I updated to a newer kernel and now the fanotify watch works as expected.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

file system watching on linux fails to avoid placing marks from different mounts into the same fanotify descriptor
3 participants